Artificial neural networks generally consist of multiple layers of neuron devices which provide massively parallel computing power. An interesting feature of such networks is their adaptive capabilities which allow the network to learn new information. These characteristics provide parallel processing of information at high computational rates--far exceeding the performance of conventional von Neumann computers which execute a program of instructions sequentially.
Neural networks generally take the form of a matrix of connections which simulate the function of a biological nervous system. Typically, electrical circuits are employed to provide variable strength synaptic connections between a plurality of inputs and a number of summing elements (i.e., neurons). The strength of the interconnections is commonly referred as the "weight" of the network. The synaptic weight, which frequently changes during the training or learning process, basically modulates the amount of charge or voltage input into each neuron within the network.
In the past, electrical synapse cells which employ floating gate devices have been used for storing connection weights in the form of electrical charge. In a floating gate device, current flow is modulated in a way which depends upon the value of the stored electrical charge. In these cells, a dot product calculation is normally performed wherein an applied input voltage is multiplied by the stored weight to produce an output. This output is then summed with other synaptic outputs in the network. Examples of semiconductor synapse cells which employ floating gate devices for storing weights are found in U.S. Pat. Nos. 4,956,564 and 4,961,002.
Another category of neural network computes the euclidean distance between the input value and stored weight value. This type of network calculates what is frequently referred to as a "city block" difference calculation. A city block difference calculation is performed by taking the absolute difference between an applied input and the stored weight of the cell. As in the case of synapse cells which calculate dot products, the output of a distance calculating synapse is usually summed with other similar outputs within the network. Both the multiplying and difference calculating types of neural networks are equally capable of solving computational tasks such as associative memory and pattern classification.
As will be seen, the present invention discloses a novel neural network architecture for parallel calculation of the absolute "city block" difference between an input voltage and a stored weight vector. The invented network utilizes switched-capacitor circuitry and floating gate devices for emulating the function of a biological synapse.